Weight-decay regularization in Reproducing Kernel Hilbert Spaces by variable-basis schemes

The optimization problems associated with various regularization techniques for supervised learning from data (e.g., weight-decay and Tikhonov regularization) are described in the context of Reproducing Kernel Hilbert Spaces. Suboptimal solutions expressed by sparse kernel models with a given upper bound on the number of kernel computational units are investigated. Improvements of some estimates obtained in Comput. Manag. Sci., vol. 6, pp. 53-79, 2009 are derived. Relationships between sparseness and generalization are discussed. Key–Words: Learning from data, regularization, weight decay, suboptimal solutions, rates of approximation.